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I.  INTRODUCTION 


A.  BACKGROUND 

Prediction  markets — also  known  as  information  markets,  internal  markets, 
decision  markets,  or  futures  markets — collect  and  gather  information  from  a  number  of 
people  to  make  a  prediction  about  a  future  event.  Information  is  gathered  by  buying  and 
selling  shares  in  alternative  outcomes  for  some  future  event.  The  idea  is  that  every 
person  in  the  market  has  a  small  piece  of  information  about  the  future  event,  and  the  most 
likely  outcome  will  be  identified  when  information  is  aggregated  via  a  prediction  market. 
Traders  use  their  best  information  when  buying  and  selling  contracts  because  incentives 
are  tied  to  accurate  predictions,  not  biased  expectations.  “In  a  truly  efficient  prediction 
market,  the  market  price  will  be  the  best  predictor  of  the  event,  and  no  combination  of 
available  polls  or  other  information  can  be  used  to  improve  on  the  market-generated 
forecasts”  (Wolfers  &  Zitzewitz,  2004,  p.  108).  From  the  prediction  market  outcome, 
management  can  make  strategic  decisions  about  future  sales  or  products. 

The  range  of  current  applications  includes  political  elections,  entertainment, 
sports,  science  and  technology  developments,  commodities  futures,  current  events,  and 
economic  data.  “[Prediction  markets]  are  practical  for  corporate  strategists  given  that 
internal  markets  can  stream  decentralized  intelligence  into  accurate  predictions  about 
future  events”  (Corporate  Executive  Board,  2006,  p.  2).  A  number  of  companies 
currently  use  prediction  markets,  including  Hewlett-Packard  (HP),  Microsoft,  Google, 
Yahoo,  Siemens,  Eli  Lilly,  and  others. 

The  Department  of  Defense  (DoD)  attempted  to  use  prediction  markets  in  the  past 
but  was  met  with  political  pressure  to  discontinue  their  use.  After  the  intelligence 
community’s  failure  to  efficiently  aggregate  intelligence  information  prior  to  the 
September  11,  2001  terrorist  attacks,  the  Defense  Advanced  Research  Projects  Agency 
(DARPA)  created  a  Policy  Analysis  Market  (PAM)  to  predict  terror  attacks  and  other 
political  events  around  the  world.  PAM  was  subsequently  cancelled  in  2003,  one  day 
after  it  was  announced.  The  controversy  with  PAM  arose  from  the  concern  that 
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individuals  could  manipulate  and  profit  from  this  market,  including  potential  terrorists. 
The  goal  of  this  thesis  is  not  to  predict  political  or  terror  events,  but  to  provide 
management  with  an  additional  forecasting  tool  to  make  informed  decisions  about 
programs  and  projects. 

B.  BENEFIT  OF  THESIS 

With  the  current  and  future  pressures  on  the  Defense  budget,  this  thesis  will 
investigate  the  benefits  and  limitations  of  prediction  markets  as  an  additional 
(complementary)  tool  to  manage  a  portfolio  of  acquisition  programs.  The  potential  role 
of  prediction  markets  in  portfolio  management  would  be  to  help  forecast  which  programs 
are  most  likely  to  succeed,  where  success  means  that  the  program  is  on  or  under  budget, 
on  or  ahead  of  schedule,  and  meeting  technical  performance  requirements  or  objectives. 
In  addition,  success  means  viability  in  the  real  world  (i.e.,  would  the  end  product  be 
effective  for  the  end  user).  Prediction  markets  have  been  extensively  investigated  in 
controlled  settings,  and  private  sector  companies  have  used  prediction  markets  in  the  past 
to  help  forecast  the  potential  market  success  of  products  in  their  research  and 
development  (R&D)  portfolios.  This  thesis  will  review  the  applications  and  results  of 
prediction  markets  in  such  environments,  and  deduce  if  and  how  the  DoD  can  best 
implement  prediction  markets  as  a  management  tool. 

C.  THESIS  SCOPE 

Private  sector  companies  have  used  prediction  markets  to  help  manage  research 
and  development  portfolios,  employing  these  prediction  markets  to  forecast  which 
products  will  achieve  technical  objectives,  be  successful  at  market,  as  well  as  avoid  cost 
and  schedule  overruns.  Current  and  future  budget  constraints  have  led  to  much 
discussion  on  cutting  certain  acquisition  programs  in  the  DoD.  Prediction  markets  could 
potentially  be  used  to  assist  with  decisions  regarding  which  programs  to  cut  based  on 
potential  success,  and  cost  and  schedule  overruns.  This  thesis  explores  the  feasibility  of 
using  prediction  markets  as  a  management  tool  in  the  DoD.  By  reviewing  past  prediction 
market  applications  and  results,  the  research  will  analyze  if  and  how  prediction  markets 
can  best  be  used  in  the  DoD  acquisition  process. 


2 


D.  METHODOLOGY 


The  approach  to  this  thesis  will  primarily  focus  on  reviewing  information 
currently  available  on  prediction  markets.  The  literature  review  will  investigate: 

•  How  and  why  prediction  markets  work  in  principle 

•  How  the  performance  or  “success”  of  prediction  markets  is 

measured 

•  The  market  design  elements  that  must  be  chosen  and  calibrated  in 
developing  and  implementing  prediction  markets 

•  How  prediction  markets  are  applied  in  private  sector  companies 

•  How  prediction  markets  are  applied  to  forecast  the  outcome  of 

other  “real  world”  events  (elections,  sporting  contests,  motion 
picture  success) 

The  thesis  will  then  make  a  connection  between  these  previous  prediction  market 
applications  and  results  to  the  DoD.  Specifically,  the  primary  objective  is  to  determine 
how  DoD  can  use  prediction  markets  to  manage  a  portfolio  of  acquisition  programs. 

E.  ORGANIZATION  OF  THESIS 

The  thesis  begins  with  an  overview  of  prediction  markets;  specifically,  what  they 
are  and  why  they  are  valuable.  Next,  the  market  elements  and  decision  variables  in 
prediction  market  design  are  investigated.  The  decision  variables  play  a  key  role  in 
prediction  market  success  or  failure.  Then,  the  thesis  discusses  measures  of  performance 
or  success  of  a  prediction  market.  This  can  prove  difficult  in  real  world  environments. 
Prediction  markets  measure  probabilities  of  different  outcomes.  If  a  prediction  market 
indicates  one  outcome  has  a  higher  probability  than  another  alternative  outcome,  but  the 
lower  probability  event  occurs,  it  does  not  mean  that  the  prediction  market  misjudged  the 
probabilities;  low  probability  events  do  occur. 

Furthermore,  prediction  markets  can  only  be  as  accurate  as  the  information 
available  to  the  market  participants.  The  fact  that  the  market  gets  it  wrong  may  simply 
indicate  the  information  available  was  low  quality.  Similarly,  accurate  prediction  market 
outcomes  may  simply  indicate  such  an  abundance  of  accurate  information  that  any 
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forecast  could  be  correct.  The  most  relevant  test  for  prediction  market  performance  is 
whether  repeated  prediction  market  outcomes  more  accurately  reflect  the  probabilities  of 
actual  outcomes  than  alternative  forecasting  techniques. 

Private  sector  applications  and  results  are  evaluated,  along  with  other  “real  world” 
applications  and  results.  Finally,  the  thesis  connects  prediction  market  applications  to  the 
DoD.  Specifically,  how  might  the  DoD  use  prediction  markets  to  aid  in  controlling 
acquisition  costs  and  schedule  overruns. 


4 


II.  PREDICTION  MARKETS:  WHAT  THEY  ARE  AND  WHY 
THEY  ARE  (POTENTIALLY)  VALUABLE 

Prediction  markets,  in  their  simplest  form,  collect  and  gather  information  from  a 
number  of  people  to  make  a  prediction  about  a  future  event.  The  idea  is  that  each  person 
has  a  small  piece  of  information,  and  by  collecting  and  aggregating  all  the  information, 
each  can  make  the  most  accurate  possible  prediction.  Predicting  the  outcome  of 
uncertain  events  like  political  or  sporting  events,  sales  forecasts,  or  macroeconomic 
indicators  clearly  requires  that  we  have  relevant  and  sufficient  information  with  which  to 
make  our  predictions  as  accurate  as  possible.  Information  that  is  typically  relevant  is  not 
concentrated  in  the  hands  of  one  person,  or  even  a  few  people.  Instead,  it  is  distributed 
between  many  people,  each  of  whom  is  likely  to  have  a  small  bit  of  pertinent  knowledge. 
An  optimal  solution  would  be  to  ask  all  these  people  to  share  what  they  know  and  to 
aggregate  all  the  information  they  have.  To  “poll”  every  individual  would  require  a  lot  of 
time,  effort,  and  money,  so  this  method  is  not  feasible.  A  prediction  market  is  a  relatively 
low  cost  and  efficient  means  of  “polling”  a  large  number  of  individuals. 

The  following  is  a  simple  example  of  how  information  aggregation  is  beneficial 
(Kalovcova,  2007):  Let  us  assume  there  are  six  possible  states  A,  B,  ...,  F,  all  with  the 
same  initial  probability  of  occurring.  Five  individuals  receive  a  private  signal.  The 
signal  is  drawn  from  an  urn  where  the  true  state  (for  this  example,  we  assume  it  is  ‘A’)  is 
represented  by  five  balls  and  the  remaining  five  states  (B  through  F)  are  represented  by 
two  balls  each.  The  probability  of  drawing  the  correct  signal  is  therefore  one  third  (five 
out  of  fifteen),  and  the  probability  of  drawing  the  wrong  signal  is  two  thirds  (ten  out  of 
fifteen).  Each  individual  draws  three  balls,  with  the  following  results: 

AAB  AEE  ABF  ACD  CDF 

1  1  i  i  1 

A  E  ?  ?  ? 

The  first  individual  thinks  the  true  state  is  ‘A’  and  is  right,  the  second  thinks  the  true  state 
is  ‘E’  and  is  wrong,  and  the  remaining  three  individuals  have  no  ability  to  predict  the  true 
state  based  on  the  signal  they  received,  but  think  any  of  three  possible  states  are  equally 
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likely.  The  aggregated  information  (AAAAA,  BB,  CC,  DD,  EE,  and  FF)  provides  a 
much  stronger  signal  about  the  true  state  compared  to  the  private  signal  of  any  single 
individual.  The  problem  is  the  difficulty  in  finding  all  the  individuals  who  possess  some 
relevant  information,  and  encouraging  individuals  to  reveal  their  signals  truthfully.  For 
both  reasons,  prediction  markets  are  used  as  a  mechanism  for  individuals  to  truthfully 
reveal  their  information. 

Prediction  markets  usually  provide  quite  accurate  forecasts,  and  have  typically 
outperformed  alternative  prediction  tools.  One  key  benefit  of  markets  is  that  they 
motivate  traders  to  re-evaluate  the  quality  of  their  information  on  an  ongoing  basis 
because  trading  continues  as  new  information  is  revealed. 

The  power  of  prediction  markets  derives  from  the  fact  that  they  provide 
incentives  for  truthful  revelation,  they  provide  incentives  for  research  and 
information  discovery,  and  the  market  provides  an  algorithm  for 
aggregating  opinions.  (Wolfers  &  Zitzewitz,  2004,  p.  121) 
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III.  DESIGN  ISSUES/DECISION  VARIABLES  OF  PREDICTION 

MARKETS 

The  success  of  prediction  markets,  like  any  market,  depends  on  their  design  and 
implementation.  Some  design  issues  include  how  buyers  are  matched  to  sellers  (market 
design),  the  specification  of  the  contract  (contract  type),  whether  real  money  is  used 
(payoff  type),  and  whether  enough  information  exists  within  the  participant  pool  to 
provide  a  basis  for  trading  and  support  accurate  predictions  (participant  quality). 

A.  MARKET  DESIGN 

Most  prediction  markets  match  buyers  and  sellers  through  a  continuous  double 
auction  mechanism.  Buyers  submit  bids  and  sellers  submit  asking  prices,  and  the 
mechanism  executes  a  trade  whenever  the  two  sides  reach  a  mutually  agreeable  price; 
this  market  design  requires  a  large  bidder  pool  so  that  buyers  and  sellers  can  be  readily 
matched.  Another  method  is  the  pari-mutual  system,  where  all  of  the  money  that  is  bet 
goes  into  a  common  pot  and  is  then  divided  among  the  winners.  This  system  is  common 
in  horse  race  betting.  Lastly,  markets  may  also  use  a  market  maker  who  announces 
willingness  to  buy  and  sell  at  a  certain  range  of  prices.  The  market  maker  determines 
price  using  a  pricing  algorithm  and  executes  trades  regardless  of  whether  there  are  buyers 
and  sellers  on  both  sides  of  the  market.  This  mechanism  is  beneficial  when  liquidity  in 
the  market  is  limited  and  places  a  great  deal  of  risk  on  the  market  maker. 

B.  CONTRACT  TYPE 

A  prediction  market  motivates  people  to  make  predictions  based  on  their  best 
information.  For  the  markets  to  work  well,  the  contracts  must  be  well  defined,  easily 
understandable,  and  tied  to  a  specific  event  occurring  at  some  fixed  point  in  time. 

Table  1  summarizes  the  three  main  types  of  contracts  in  prediction  markets.  First, 

in  a  “winner-take-all”  contract,  the  contract  costs  $p,  where  $p  <  1 ,  and  pays  off  $1,  if 

and  only  if  a  specific  event  occurs.  The  price  of  a  winner-take-all  market  represents  the 

market’s  expectation  of  the  probability  that  the  event  will  occur.  Second,  an  “index” 

contract  price  provides  payoffs  that  vary  continuously  based  on  an  index  number,  like  the 
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number  of  units  sold  or  percent  of  the  total  votes  received.  Bidders  purchase  contracts  if 
they  expect  the  final  outcome  to  exceed  the  level  indicated  by  the  current  price,  and  sell 
shares  if  they  expect  the  fixed  outcome  to  fall  short  of  the  currently  predicted  level.  The 
price  of  an  index  contract  represents  the  market’s  mean  expected  value  of  the  outcome. 
Finally,  in  “spread”  betting,  traders  differentiate  themselves  by  bidding  on  the  cutoff  that 
determines  whether  an  event  occurs,  like  whether  sales  volume  will  exceed  a  certain 
threshold.  The  outcome  of  a  spread  market  can  yield  the  market’s  expectation  of  the 
median  outcome. 

Additionally,  a  family  of  prediction  markets  can  be  used  to  evaluate  uncertainty 
of  the  expectations  of  the  event.  For  instance,  a  family  of  winner-take-all  contracts  that 
pay  off  only  if  the  sales  volume  is  either  less  than  10  units,  greater  than  10  but  less  than 
20  units,  greater  than  20  but  less  than  30  units,  and  so  on,  reveals  almost  the  entire 
probability  distribution  of  the  market’s  expectation.  Experience  at  Microsoft  has  shown 
that  using  multiple  contracts,  and  revealing  the  market’s  probability  distribution,  results 
in  the  most  accurate  prediction  (Corporate  Executive  Board,  2006).  Figure  1  illustrates 
that  a  market  using  multiple  contracts  results  in  a  probability  distribution  of  the  most 
likely  range  of  outcomes. 


Figure  1.  Multiple  Contract  Results  (Illustrative  Example) 
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Table  1.  Contract  Types  (From  Wolfers  &  Zitzewitz,  2004) 


Contract 

Example 

Details 

Reveals  market 
expectation  of... 

Winner-take-all 

Event:  Sales  volume 
is  greater  than  X  units. 

Contract  pays  $1  if 

event  occurs. 

Probability  that  event 
will  occur. 

Index 

Contract  pays  $  1  for 
every  unit  sold. 

If  5 1  units  are  sold, 
the  contract  pays  $5 1 . 

Mean  value  of 
expected  outcome. 

Spread 

Contract  pays 
additional  money  if 
sales  volume  is  more 
than  a  given 
threshold. 

Contract  costs  $1.  If 
spread  is  fixed  at  45 
units  and  sales  are 

>45  units,  contract 
pays  $2.  If  not, 
contract  pays  nothing. 

Median  value  of 
expected  outcome. 

C.  PAYOFF  TYPE 

The  appropriate  incentive  structure  must  also  exist.  The  market  may  be  designed 
well  with  appropriate  trading  mechanisms  and  contract  types,  but  may  fail  if  there  is  no 
motivation  to  trade.  This  raises  the  question  of  whether  to  use  real  money  or  play  money 
markets.  With  play  money,  participants  are  provided  an  initial  endowment  and  are 
presumed  to  be  motivated  to  maximize  the  final  value  of  this  endowment  after  a 
predetermined  trading  window.  With  real  money,  participants  may  be  given  an  initial 
endowment  of  real  money,  or  they  may  be  expected  to  invest  their  personal  funds;  they 
are  presumably  motivated  to  earn  as  much  money  as  possible  over  the  trading  window. 
According  to  Wolfers  and  Zitzewitz  (2004),  there  has  been  no  difference  in  the 
performance  of  real  versus  play  money  markets.  Some  early  adopters  indicated 
employee  resistance  to  prediction  markets  because  they  disagreed  with  the  idea  of  market 
incentives.  Critics  argue  that  prediction  markets  generate  predictions  that  a  company 
presumes  its  employees  are  supposed  to  provide  anyway  (Corporate  Executive  Board, 
2006).  For  instance,  at  Beta  the  toughest  thing  was  getting  agreement  on  the  incentive 
structure.  An  IT  Research  Manager  at  Beta  states  it  best: 
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A  group  of  people  will  think  that  monetary  incentives  are  worthwhile 
given  the  potential  uses  and  benefits  of  market  predictions.  But,  on  the 
flipside,  another  group  believes  that  incentives  are  inappropriate,  taking 
issue  with  paying  people  for  their  knowledge  when  forecasting  is  a  part  of 
the  job  they  do. 

The  goal  of  the  incentive  structure  is  to  encourage  market  participation  to  achieve  the 
most  accurate  result  possible.  To  aggregate  all  the  information  available,  participation  by 
those  with  the  knowledge  is  necessary,  and  the  participants  must  be  motivated  to  trade. 

D.  PARTICIPANT  QUALITY 

Another  decision  factor  is  selecting  the  traders.  Participants  need  to  be  selected 
carefully  to  ensure  no  one  with  relevant  information  is  overlooked.  The  mix  of  traders 
needs  to  be  diverse,  so  that  people  bring  different  pieces  of  information  to  the  table. 
Additionally,  one  must  decide  whether  to  include  uninformed  traders  in  the  market  or  to 
allow  only  informed  participants.  Uninformed  traders  are  market  participants  who  have 
little  or  no  information  regarding  the  future  event  on  which  the  market  focuses. 
Uninformed  traders  expand  the  participant  pool  and  generate  liquidity.  The  downside  to 
uninformed  traders  is  their  potential  lack  of  interest  in  participating.  Designers  at  Beta 
and  Siemens  strived  for  a  diverse  group  of  participants  in  their  markets,  including 
employees  from  all  divisions  of  the  company. 

Along  with  trader  selection,  educating  participants  is  just  as  important.  It  is 
important  for  traders  to  re-evaluate  their  information  often.  “Inexperienced  traders  tend 
to  make  one  trade  in  the  early  stages  of  the  market  and  then  not  participate  for  the 
remainder  of  the  time.  Veteran  traders  participate  actively  by  gathering  new  information 
from  the  market,  monitoring  the  trading  behavior  of  others,  and  re-evaluating  their  own 
information”  (Corporate  Executive  Board,  2006).  The  most  practical  way  to  train 
participants  is  to  set  up  experimental  markets  so  traders  can  figure  out  how  the  market 
works  and  gain  the  experience  necessary  to  be  better  participants.  Research  has  also 
shown  that  market  prediction  accuracy  increases  through  additional  repetitions  as 
participants  gain  experience  in  how  prediction  markets  work. 
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IV.  MEASURING  PERFORMANCE  OR  SUCCESS 


The  following  features  make  markets  more  effective  compared  to  traditional 

tools. 

A.  ANONYMITY 

When  a  manager  asks  a  direct  report  to  predict  a  business  outcome  or  whether  a 
project  will  be  successful,  the  response  is  invariably  impacted  by  what  the  manager  wants 
to  hear.  Anonymity  frees  traders  from  this  constraint. 

B.  INCENTIVES 

Market  incentives  mean  that  participants  trade  on  their  best  information  and  what 
they  truly  believe  will  happen.  This  greatly  reduces  the  effects  of  bias  found  in 
traditional  forecast  methods. 

C.  RANGE  OF  OUTCOMES 

A  well-run  prediction  market  includes  contracts  tied  to  both  positive  and  negative 
outcomes.  This  forces  the  recognition  that  failure  is  possible,  and  if  the  market  indicates 
that  failure  is  likely,  the  organization  can  adjust  accordingly. 

D.  REAL-TIME  UPDATES 

Contract  prices  update  in  real-time  and  reflect  traders’  shifting  beliefs  as  they 
obtain  new  information.  This  is  in  contrast  to  other  tools,  such  as  surveys,  which  produce 
static  data  and  must  be  rerun  to  obtain  new  information. 

For  these  reasons,  markets  are  generally  more  accurate  than  traditional  forecasts, 
and  even  inaccurate  markets  provide  valuable  information.  Markets  are  as  useful  for 
what  they  reveal  about  employee  beliefs  as  for  their  predictions.  The  fact  that  employees 
are  irrationally  optimistic  or  pessimistic  is  useful  in  and  of  itself. 
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It  is  important  to  realize  that  the  markets  will  always  tell  you  something,  whether 
they  are  accurate  or  not.  You  may  learn  about  the  perceptions  of  employees,  or  simply 
that  there  was  not  enough  information  to  gamer  an  accurate  forecast.  Regardless,  the 
outcomes  of  a  market  prediction  should  not  be  thrown  to  the  wayside.  Markets  gauge  the 
sentiment  of  traders,  and  whether  employees  support  official  plans  and  forecasts.  “Even 
when  market  predictions  are  inaccurate,  executives  can  use  trader  behavior  to  understand 
organizational  alignment  and  employee  engagement”  (Corporate  Executive  Board,  2006). 
This  allows  executives  the  opportunity  to  make  necessary  changes  to  strategic  and 
operational  plans. 

For  Microsoft,  the  biggest  surprise  was  how  little  some  people  cared  about 
the  accuracy  of  internal  markets.  What  actually  sold  most  on  these 
markets  was  their  ability  to  aggregate  opinions  quickly  into  an  easily 
decipherable  set  of  predictions.  Even  if  the  predictions  were  wrong,  the 
point  was  that  this  was  the  outcome  that  traders  believed,  and  knowing 
that  was  very  useful.  (Todd  Proebsting,  Microsoft) 

It  is  important  to  look  at  the  results  to  determine  what  they  are  providing,  and 
whether  the  predictions  are  accurate. 
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V.  PRIVATE  SECTOR  APPLICATIONS  AND  RESULTS 


The  current  corporate  applications  for  internal  prediction  markets  include  sales 
forecasting,  project  execution,  product  design,  trend  forecasting,  and  resource  allocation. 
Sales  forecasting  predicts  the  likely  volume  of  sales  in  dollars  or  units.  Project  execution 
predicts  when  projects  will  reach  their  planned  milestones.  Product  design  forecasts 
which  product  features  or  enhancements  customers  will  prefer.  Trend  forecasting  reveals 
new  or  existing  market,  technology,  or  customer  trends.  Resource  allocation  enables 
business  units  to  trade  resources  according  to  their  needs,  and  can  be  used  to  support 
objectives  such  as  corporate  social  responsibility  (Corporate  Executive  Board,  2006). 


Table  2.  Experiences  of  Early  Adopters  (From  Malone,  2004  &  Kiviat,  2004) 


Application 

Practitioner 

Description 

Sales 

Forecasting 

Hewlett- 

Packard 

Hewlett-Packard  used  an  internal  market  system  to 
forecast  printer  sales  with  considerable  accuracy.  Front 
line  sales  employees  exchanged  contracts  representing 
the  future  sales  volume  based  on  their  predictions  of 
future  printer  sales.  When  trading  ended,  the  contract 
valued  most  highly  represented  the  most  likely  sales 
range.  HP’s  official  forecast  erred  by  13%,  while  the 
market  erred  by  6%.  In  further  trials,  the  market 
performance  exceeded  the  accuracy  of  official  forecasts 
75%  of  the  time. 

Product 

Development 

Eli  Lilly 

Eli  Lilly  applied  internal  markets  to  predict  correctly 
which  of  six  potential  new  drugs  would  have  the 
greatest  success  in  passing  product  development 
hurdles.  Employees  involved  in  different  stages  of  drug 
development  traded  market  contracts  based  on  their 
information.  The  market  aggregated  information  with 
accuracy  and  opinion  detail  that  would  not  have 
emerged  had  traders  responded  to  a  poll. 

General 

Forecasting 

Google 

Google  uses  internal  markets  to  forecast  events  such  as 
new  product  launch  dates  and  new  office  openings.  The 
company  applies  market  predictions  to  determine  the 
likelihood  that  an  event  will  occur  and  on  a  specific 
date. 
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Social 

Responsibility 

BP 

BP  set  up  an  internal  market  to  reduce  the  oil 
company’s  greenhouse  gas  emissions.  Each  business 
unit  was  granted  the  right  to  generate  a  certain  amount 
of  emissions  and  given  access  to  an  electronic  trading 
system  that  allowed  it  to  buy  more  emissions  capacity 
or  sell  excess  capacity  to  other  units.  The  market 
approach  enabled  BP  to  achieve  its  targeted  reductions 
nine  years  ahead  of  schedule. 

Project 

Milestones 

Siemens 

Siemens  used  internal  prediction  markets  to  predict 
software  project  milestones.  On  one  occasion, 
traditional  methods  suggested  a  software  project  would 
be  delivered  on  time,  but  the  prediction  market 
suggested  it  would  be  2-3  weeks  delayed.  The  project 
turned  out  to  be  1 1  workdays  late. 

Project 

Milestones 

Microsoft 

Microsoft  uses  internal  markets  to  predict  whether 
projects  will  meet  milestones  articulated  in  their  project 
plans. 

A.  HEWLETT-PACKARD 

An  internal  market  at  HP  produced  more  accurate  forecasts  of  printer  sales  than 
the  firm’s  internal  processes  (Chen  &  Plott,  2002).  A  total  of  12  predictions  were 
performed  over  a  period  of  three  years.  The  prediction  markets  at  HP  included 
predictions  for  eight  products.  In  some  cases,  dollar  sales  were  predicted,  and  in  other 
cases,  the  number  of  units  sold  was  predicted.  The  market  design  employed  at  HP  was 
the  web-based  double  auction  market  of  Marketscape  software,  developed  at  the 
Laboratory  of  Economics  and  Political  Science  at  Caltech  (Chen  &  Plott,  2002).  From 
the  web  interface,  participants  could  enter  a  buy  offer,  a  sell  offer,  or  acceptance  of  an 
offer.  If  a  trade  was  possible,  it  was  executed;  if  not,  the  order  was  placed  in  an  order 
book.  According  to  Plott  and  Sunder  (1988),  experiments  involving  single  compound 
securities  can  have  difficulty  with  information  aggregation.  As  such,  HP  decided  to  use  a 
complete  set  of  state  contingent  contracts  (Chen  &  Plott,  2002).  For  example,  if  HP 
wanted  to  predict  future  sales  of  a  product,  they  would  establish  a  prediction  market  with 
multiple  securities  with  each  having  an  interval  of  say  100  units.  Therefore,  there  would 

be  a  security  for  0-100  units  sold,  101-200  units  sold,  and  so  on.  Depending  on  the 
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interval  in  which  the  final  outcome  falls,  the  corresponding  security  pays  one  dollar  per 
share;  all  other  securities  pay  nothing.  The  payoff  for  HP  markets  involved  real  money  in 
which  the  “winning”  security  paid  off  a  fixed  amount;  all  other  securities  paid  nothing. 
HP  had  issues  engaging  employees  to  participate  in  an  activity  in  which  they  may  lose 
money  (Chen  &  Plott,  2002).  Thus,  HP  supplemented  participants  with  money  at  the 
beginning  of  the  market  sessions  to  ensure  participation  and  minimize  the  potential 
employee  loss.  The  markets  at  HP  typically  included  20-30  people,  mostly  from  the 
marketing  and  finance  divisions  (Chen  &  Plott,  2002).  Additionally,  about  five 
participants  were  from  HP  Labs,  who  had  little  or  no  information  about  the  predicted 
event,  but  provided  additional  liquidity  to  the  market. 

The  results  were  that  the  internal  market  predictions  were  closer  to  the  actual 
outcome  than  the  official  forecast  in  six  out  of  eight  events  (Chen  &  Plott,  2002).  These 
results  are  very  promising  for  using  prediction  markets  to  predict  future  sales,  compared 
to  traditional  forecasting. 

B.  SIEMENS 

Siemens  has  used  prediction  markets  for  software  projects.  Ortner  (1998) 
describes  an  experiment  at  Siemens  in  which  an  internal  market  predicted  the  firm  would 
fail  to  deliver  a  software  project  on  time  even  when  traditional  planning  tools  suggested 
the  deadline  could  be  met.  The  Siemens  market,  like  HP,  used  a  fully  computerized 
double  auction  market  with  a  software  product  called  FX  developed  by  Kumo,  Inc. 
(Ortner,  1998).  For  this  software  project,  Siemens  created  two  separate  prediction 
markets.  One  asked  a  simple  question:  “Can  the  project  be  finished  in  the  planned  time 
horizon?”  The  payoff  rule  was  a  simple  winner-take-all  design.  The  current  market  price 
for  this  security  predicts  the  probability  of  meeting  the  planning  time  horizon.  The 
second  market  was  designed  to  predict  the  length  of  the  possible  delay.  This  market 
included  two  shares  called  “Early”  (or  YES)  and  “Late”  (or  NO).  The  payoff  structure 
for  this  market  was  set  up  in  a  linear  fashion.  The  YES  share  paid  the  maximum  of  (1— 
0.2*weeks  late)  or  zero.  The  NO  share  paid  the  minimum  of  (0.2*weeks  late)  or  one. 
For  example  if  the  price  of  the  YES  share  was  trading  at  0.8,  the  market  believes  there 
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will  be  a  one-week  delay.  Conversely,  if  the  NO  share  was  trading  at  0.2,  the  market 
believes  there  will  be  a  one-week  delay.  This  payoff  structure  is  illustrated  in  Figure  2. 

price 


Figure  2.  Pay-off  Structure  at  Siemens  (From  Ortner,  1998) 

Siemens  opened  the  market  to  all  people  working  in  the  project  except  upper  level 
management.  Sixty-three  traders  joined  the  market  and  about  50  became  active  traders. 
Of  the  participant  pool,  67%  were  developers,  31%  group  managers,  and  2%  project 
managers  (second  level-first  level  managers  were  not  allowed  to  join  the  experiment, 
because  of  their  manipulating  power)  (Ortner,  1998).  The  Siemens  market  did  not  use 
any  uninformed  traders. 

Results:  Initially,  after  opening  the  two  markets,  the  winner-take-all  market  YES 
shares  approached  a  price  of  0.43  and  fluctuated  between  0.43  and  0.40  for 
approximately  six  weeks.  About  one  month  prior  to  the  deadline,  the  YES  shares  for  the 
winner-take-all  market  plummeted  indicating  the  market  did  not  believe  the  project 
would  reach  its  planned  milestone,  although  it  was  still  possible  according  to  the 
traditional  project  plan  used  by  the  management  team  (Ortner,  1998).  In  the  end,  the 
market  was  closed  when  the  project  manager  announced  the  milestone  time  limit  was  not 
reached.  Therefore,  each  YES  share  paid  zero  and  the  NO  shares  paid  one.  In  the  second 
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market,  used  to  predict  the  time  delay,  after  only  one  month  of  trading  and  more  than 
three  months  before  the  scheduled  deadline,  the  market  predicted  a  delay  of  two  to  three 
weeks.  In  the  end,  the  actual  delay  turned  out  to  be  11  workdays. 

HP  and  Siemens’  experiences  suggest  that  motivating  employees  to  trade  is  a 
major  challenge.  Both  firms  ran  real  money  exchanges,  with  a  relatively  small  trading 
population  (20-60  people),  and  subsidized  market  participation  by  either  providing 
traders  with  a  portfolio  or  matching  initial  deposits.  Even  with  the  subsidies  and  small 
trading  population,  the  predictive  performance  of  these  markets  was  remarkable. 

C.  MICROSOFT 

Microsoft  uses  internal  markets  to  predict  whether  projects  will  meet  milestones 
articulated  in  their  project  plans.  Microsoft’s  market  relies  on  an  automated  market 
maker  that  enables  traders  to  access  the  market  at  their  convenience  to  buy  and  sell 
contracts.  By  using  a  market  maker,  traders  can  exchange  contracts  without  relying  on 
others’  willingness  to  buy  or  sell.  Microsoft  uses  multiple  contracts,  each  representing  a 
different  predicted  date  on  which  a  project  will  reach  a  certain  milestone.  Microsoft  also 
has  run  experimental  markets  in  the  past  involving  naive  and  informed  traders.  “Naive 
traders  did  not  impact  the  accuracy  of  market  predictions  because  informed  traders 
corrected  market  price  fluctuations  caused  by  naive  participants”  (Corporate  Executive 
Board,  2006).  However,  Microsoft  currently  limits  participation  to  informed  traders 
because  uninformed  traders  are  less  likely  to  participate.  Microsoft  selects  its  traders  by 
targeting  employees  who  have  enough  information  to  make  educated  trades,  and  selecting 
traders  from  different  corporate  functions  to  aggregate  different  types  of  information, 
giving  more  accurate  results  (Corporate  Executive  Board,  2006).  The  only  concern  is 
leaving  out  someone  who  has  information,  but  management  is  unaware  of  their  having 
sufficient  information.  Microsoft  encourages  participants  to  trade  when  they  think  they 
can  contribute  to  the  market. 
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Table  3.  Summary  of  Private  Sector  Applications 


Application 

Practitioner 

Market 

Design 

Contract 

Type 

Payoff 

Type 

Participant 

Quality 

Measure  of 
Success 

Sales 

Forecasting 

Hewlett- 

Packard 

Double 

Auction 

Multiple 

Contracts 

Real 

money 

20-30  from 
marketing 
and  finance 
divisions 

six  of  eight 
events  were 

closer  to 
actual 

outcome  than 
official 
forecast. 

Product 

Development 

Eli  Lilly 

NA 

NA 

NA 

NA 

NA 

General 

Forecasting 

Google 

NA 

NA 

NA 

NA 

NA 

Social 

Responsibility 

BP 

NA 

NA 

NA 

NA 

NA 

Project 

Milestones 

Siemens 

Double 

Auction 

Winner- 

take-all 

Real 

money 

50-60  from 
all  aspects  of 
a  project 
except  upper 
level 

management 

Predicted 
time  delay  of 
2-3  weeks. 
Actual  delay 
was  11 
workdays. 

Project 

Milestones 

Microsoft 

Automated 

Market 

Maker 

Multiple 

Contracts 

NA 

Limits 
participation 
to  informed 
traders. 

NA 
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VI.  OTHER  “REAL  WORLD”  APPLICATIONS  AND  RESULTS 


A.  IOWA  ELECTRONIC  MARKET 

The  Iowa  Electronic  Market,  run  by  the  University  of  Iowa,  is  probably  the  best- 
known  prediction  market  among  economists.  The  Iowa  Electronic  Market  uses  a  double 
auction  market  with  winner-take-all  and  index  contract  types.  It  is  a  real  money  market 
with  no  endowment,  and  participation  is  open  to  anyone  interested,  but  likely  only 
attracts  those  particularly  interested  and  aware  of  the  market’s  existence.  In  1988,  the 
original  Iowa  experiment  allowed  trades  in  a  contract  that  paid  2Vi  cents  for  each 
percentage  point  of  the  popular  vote  in  the  presidential  election  received  by  Bush, 
Dukakis,  and  others.  More  recently,  it  has  run  prediction  markets  based  on  the  2008 
presidential  election,  the  2008  congressional  elections,  and  the  2008  Minnesota  senate 
election.  The  Iowa  Electronic  Markets  has  yielded  both  very  accurate  predictions  and 
also  outperformed  large-scale  polling  organizations  (Berg,  et  al.,  2001).  Figure  3  shows 
data  from  the  four  U.S.  presidential  elections  between  1988  and  2000.  The  horizontal 
axis  shows  the  number  of  days  until  the  election,  and  the  vertical  axis  displays  the 
average  absolute  error  between  the  prediction  market  price  (linked  to  the  two-party  share 
of  the  popular  vote)  and  the  actual  popular  vote  percentage  earned  in  the  election.  In  the 
last  week  before  the  election,  the  prediction  markets  have  predicted  vote  shares  with  an 
average  absolute  error  of  approximately  1.5  percentage  points,  compared  to  the  final 
Gallup  poll  forecasts  that  differed  by  2.1  percentage  points  (Wolfers  &  Zitzewitz,  2004). 
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Figure  3.  Information  Revelation  Over  Time  (From  Wolfers  &  Zitzewitz,  2004) 

The  superior  performance  of  the  Iowa  markets  may  be  attributable  to  the  fact  that 
“traders  are  self-selected  with  a  clear  interest  in  predicting  what  will  actually  happen, 
rather  than  what  they  hope  will  happen”  (Corporate  Executive  Board,  2006).  In  a  poll, 
respondents  predict  events  without  any  context  of  others’  beliefs.  In  a  prediction  market, 
each  participant  knows  the  current  consensus  and  factors  this  information  into  decision¬ 
making. 

B.  GOLDMAN  SACHS  AND  DEUTSCHE  BANK  “ECONOMIC 

DERIVATIVES”  MARKET 

Another  example  of  the  relative  performance  of  a  prediction  market  comes  from 
the  “Economic  Derivatives”  market  established  by  Goldman  Sachs  and  Deutsche  Bank. 
This  market  is  tied  to  macroeconomic  outcomes,  such  as  non-farm  payrolls,  retail  sales, 
levels  of  the  Institute  for  Supply  Management’s  manufacturing  diffusion  index,  and 
initial  unemployment  claims  (Giirkaynak  &  Wolfers,  2005).  The  market  mechanism  is  a 
pari-mutuel  system  where  for  a  given  strike  price  all  bets  that  the  specified  outcome 
either  will  or  will  not  occur  are  pooled;  this  pool  is  then  distributed  to  the  winners  in 
proportion  to  the  number  of  options  purchased.  The  Economic  Derivatives  market  uses 
multiple  contracts,  allowing  traders  to  take  a  position  on  specified  ranges  the  data  will 

fall  in.  The  outcome  results  in  a  probability  density  function,  which  prior  to  this  market 

20 


was  unavailable.  Figure  4  compares  the  performance  of  the  Economic  Derivatives 
market  with  a  survey  of  economists  in  predicting  economic  outcomes  based  on  data 
gathered  by  Giirkaynak  and  Wolfers  (2005).  They  show  that  the  market-based  forecast 
encompasses  the  information  in  the  survey-based  forecasts.  Additionally,  the  markets’ 
response  to  data  releases  are  better  captured  in  the  market-based  expectations  than 
survey-based  expectations,  suggesting  the  markets  perform  and  react  better  than  survey- 
based  forecasting  (Giirkaynak  &  Wolfers,  2005). 


Forecasting  Economic  Outcomes 


Economic  Derivatives  Market  Forecast  □  Survey:  Average  Across  Forecasters 


o 

< 

o 

J3 

"5 

> 

"8 

i 

u 

aJ 


65- 

60- 

55- 

50- 

45- 


Business  Confidence  (ISM) 

□fti 


Initial  Unemployment  Claims 


« 


380- 

360- 

340- 

320- 

300- 


45  50  55  60 

Non-Farm  Payrolls 


65 


i — 

300 


320  340 

Retail  Sales  (ex  Autos) 


360 


400- 

AAA 

1- 

200“ 

o- 

o- 

-200- 

-400- 

•  □ 

-1- 

— i — 

100 


— i — 

200 


— r 

300 


~r~ 

-.5 


*  ® 

••• 

•  %  D 

- 1 - 

0 


T- 

.5 


Forecast  Value 


-100  0 
Graphs  by  Economic  data  series 
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Table  4  summarizes  some  of  the  more  popular  prediction  markets  available  for 
public  trade. 


Table  4.  Popular  Prediction  Markets 


Market 

Focus 

Iowa  Electronic  Markets 

<www.biz.iowa.edu/iem> 

Run  by  University  of  Iowa 

Small-scale  election  markets. 

Centrebet 

<www.centrebet.com> 

For  profit  company 

Northern  Territory  bookmaker,  offering 
odds  on  election  outcomes,  current 
events,  sports,  and  entertainment. 

Economic  Derivatives 

<www.economicderivatives.com> 

Run  by  Goldman  Sachs  and  Deutsche  Bank 

Large-scale  financial  market  trading  in 
the  likely  outcome  of  future  economic 
data  releases. 

Newsfutures 

<www.  newsfutures .  com> 

For  profit  company 

Political,  finance,  current  events  and 
sports  markets.  Also  technology  and 
pharmaceutical  futures  for  specific 
clients. 

Foresight  Exchange 

<www.ideosphere.com> 

Nonprofit  research  group 

Political,  financial,  current  events, 
science,  and  technology  events  suggested 
by  clients. 

Hollywood  Stock  Exchange 

<www.hsx.com> 

Owned  by  Cantor  Fitzgerald 

Success  of  movies,  movie  stars,  and 
awards.  Data  used  for  market  research. 

Intrade 

<www.intrade.com> 

For  profit  company 

Political,  financial,  current,  and  similar 
event  futures. 
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VII.  APPLICATION  TO  DEPARTMENT  OF  DEFENSE 
ACQUISITION  PROCESS 

The  main  DoD  application  presented  here  is  in  performance  management  of 
acquisition  programs.  Specifically,  markets  provide  insight  on  whether  the  organization 
will  meet  performance  targets  (e.g.,  schedule,  budget,  and  specifications).  If  the  market 
predicts  a  performance  gap,  resources  can  be  quickly  re-deployed  in  response,  or  kill  the 
project  to  save  time  and  money. 

One  of  the  key  problems  in  project  management  is  controlling  projects  already 
started  and  responding  to  issues  that  may  arise  unintentionally  (Ortner,  1998).  It  is  very 
important  to  receive  as  much  up-to-date  information  as  possible  on  the  progress  of  the 
project  from  within  the  group  of  people  working  on  that  project.  Project  management 
techniques,  like  milestone  trend  analysis,  should  help  recognize  delays  and  other 
problems  that  might  occur  as  soon  as  possible,  but  significant  bad  news,  like  delays, 
technical  problems,  or  budgetary  problems  are  not  often  voluntarily  passed  along  due  to 
the  fear  of  negative  feedback.  Prediction  markets  provide  a  way  for  individuals  to  pass 
information  anonymously,  which  enables  management  to  receive  timely  information  as  to 
the  status  of  a  project.  Management  can  then  react  as  necessary  to  correct  any 
deficiencies  that  may  exist.  Management  involvement  will  be  discussed  later. 

In  order  to  achieve  organizational  buy-in,  the  DoD  should  initially  position 
prediction  markets  as  an  experimental  tool.  This  approach  proves  less  threatening  than 
immediately  using  markets  as  a  replacement  tool  for  traditional  forecasting.  The  markets 
can  supplement  traditional  forecasts  and  indicate  if  the  traditional  forecasts  should  be 
revised  (Corporate  Executive  Board,  2006).  If  the  markets  are  successful,  they  may  be 
used  in  other  areas  where  traditional  methods  do  not  exist  to  provide  more  information  to 
management. 

There  are  several  criteria  that  should  be  examined  to  determine  if  a  prediction 
market  would  be  beneficial  in  a  given  application.  In  determining  whether  to  use  a 
prediction  market  or  not,  management  should  examine  if  a  contract  can  be  easily 
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defined,  if  there  is  a  known  end  point  or  date,  if  there  are  enough  participants,  and  the 
markets  should  not  include  issues  where  management  decisions  will  affect  the  market 
outcome  before  it  closes. 

A.  CONTRACT 

Defining  the  contract  is  key  to  the  market’s  success.  The  contract  should  be 
defined  in  such  a  way  that  it  is  easy  to  understand,  so  that  all  participants  know  what  they 
are  purchasing  and  what  the  pay-off  structure  is.  It  may  be  better  initially  to  limit  the 
outcomes  to  a  YES  or  NO;  for  example,  will  a  project  be  completed  on  time  or  not? 
Once  traders  are  familiar  with  how  markets  work,  then  more  complicated  markets  can  be 
devised  with  outcomes  such  as  how  many  weeks  early  or  late  will  the  project  be 
completed? 

B.  END  POINT 

Knowing  when  to  end  a  market  is  important  so  that  traders  know  there  is  a 
deadline.  Knowing  there  is  an  endpoint  will  also  provide  incentives  for  traders  to  trade. 
An  endpoint  should  be  selected  such  that  management  has  time  to  take  corrective  actions 
if  the  results  are  not  acceptable.  For  example,  the  military  would  not  want  a  market 
indicating  that  a  project  is  going  to  be  late  to  end  two  days  before  the  project’s  deadline. 
It  is  more  beneficial  for  the  market  to  end  sooner  so  that  management  can  take  action  to 
correct  deficiencies  where  needed,  or  divert  resources  to  the  project.  The  difficulty  in 
choosing  the  endpoint  is  that  the  market  must  have  a  well-defined  outcome  for  the  event 
covered  by  the  stock  when  it  ends.  If  you  directly  predict  an  event  in  which  DoD  is 
interested,  either  the  market  must  end  before  the  event,  in  which  case  you  don’t  have  the 
result,  or  the  market  close  is  too  late  to  be  useful.  One  way  markets  have  avoided  this  is 
to  predict  something  that  is  not  of  direct  management  interest  (e.g.,  projected  orders  for 
Google  so  they  could  plan  production  quantities;  the  management  decision  was 
production  plans  not  hitting  a  sales  target). 
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C.  PARTICIPANTS 

The  selection  of  traders  is  also  key  to  the  success  in  the  DoD.  As  discussed 
earlier,  the  group  should  be  diverse,  so  that  different  information  is  brought  to  the 
prediction  market.  The  traders  need  to  be  decentralized,  so  that  they  are  not  overtly 
influenced  by  upper-level  management.  In  addition,  the  selected  individuals  need  to  be 
independent,  so  that  they  pay  attention  mostly  to  their  own  information  and  do  not  worry 
about  what  everyone  around  them  thinks. 

D.  MANAGEMENT  INVOLVEMENT 

As  mentioned  above,  a  prediction  market’s  results  allow  management  to  receive 
timely  information,  and  react  to  correct  any  deficiencies  revealed  by  the  prediction 
market.  However,  management  reaction  to  what  the  prediction  market  is  predicting  prior 
to  the  market  closing  can,  and  will,  change  the  outcome  of  the  prediction.  This  would 
undermine  the  results  of  the  market,  and  discourage  users  to  participate  in  the  market. 
This  is  where  selection  of  the  end  date  becomes  important,  so  that  management  can  react 
to  the  market  without  adversely  affecting  the  market’s  results  while  still  open.  The 
securities  cannot  be  issues  in  which  management  will  intervene  as  prices  predict 
outcomes  or  the  markets  lose  integrity.  Predicting  sales  volume  helps  management  set 
production  levels,  but  if  management  increases  advertising  it  will  undermine  the  market. 
This  becomes  critical  for  the  DoD.  The  DoD  is  an  organization  that  likes  to  react  to 
current  information  available.  As  discussed,  reacting  too  early  to  information  revealed  by 
the  prediction  market  will  affect  the  outcome  of  the  market.  There  has  to  be  a 
conscientious  effort  not  to  react  immediately  to  the  information  provided  until  the  market 
ends. 
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VIII.  CONCLUSIONS 


A.  CONCLUSIONS 

Prediction  markets  may  prove  to  be  a  successful  way  to  manage  acquisition 
programs.  The  most  notable  success  documented  in  this  thesis  is  HP.  HP  ran  markets  of 
similar  types  that  could  be  used  by  the  DoD.  HP  also  showed  that  relatively  small 
markets  (approximately  20  people)  were  successful  and  provided  accurate  predictions. 

The  important  elements  for  DoD  market  design  include  defining  the  contract, 
determining  the  end  point,  selecting  participants,  and  management  involvement.  Define 
the  contract  in  such  a  way  that  it  is  easy  to  understand,  so  that  all  participants  know  what 
they  are  purchasing  and  what  the  pay-off  structure  is.  An  endpoint  should  be  selected 
such  that  management  has  time  to  take  corrective  actions  if  the  results  are  not  acceptable. 
Knowing  there  is  an  endpoint  will  also  provide  incentives  for  traders  to  trade.  The 
participation  pool  should  be  diverse,  so  that  different  information  throughout  the 
organization  is  brought  to  the  prediction  market.  The  traders  need  to  be  decentralized,  so 
that  they  are  not  overtly  influenced  by  upper-level  management.  If  the  DoD  can  design  a 
prediction  market  with  these  elements  in  mind,  and  successfully  employ  the  prediction 
market,  market  outcomes  will  provide  valuable  information  to  stake  holders  in  the  DoD. 

Additional  research  should  be  conducted  to  evaluate  (in  an  experiment  setting)  the 
effectiveness  of  these  elements  in  determining  the  accuracy  of  the  prediction  market 
outcome. 
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